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DETAILED ACTION 
Response to Arguments 

Applicant's arguments filed 6/4/07 have been fully considered but they are not 
persuasive. Firstly, applicant has argued that Luo in view of Bates is neither related to 
"visual design surfaces" nor concerned with displaying shapes on a visual design 
surface and thus, Luo is not analogous art. Examiner respectfully disagrees. It should 
be noted that the term "visual design surface" is not explicitly defined in applicant's 
disclosure, (specification p, 1-2) While applicant notes Microsoft visual studio as an 
example of an application that includes a visual design surface, with the utility of said 
design surface to model business processes and further notes visual design surfaces 
which may be commonly used \o illustrate and design systems using graphical 
notations, applicant nonetheless does not limit the term "visual design surface" to only 
these interpretations. Furthermore claims 1,12, and 19 recite representing an "element 
with a shape on a visual design surface." This is what Luo teaches. (Fig. 2, Figs. 8A- 
8B, p. 3 paragraph 35, p.4 paragraph 37, p. 4 paragraphs 37-38, paragraph 43, p.5 
paragraphs 45-48, p. 6 paragraphs 49-50) It should be noted that the element with a 
shape as taught by Luo is simply a character and furthermore a design surface as 
taught by Luo (to be visualized/displayed with an aura as the Bates reference teaches) 
is simply the background image portion of the character string that is extracted. 

Secondly applicant has argued that the Bates reference is only concerned with 
displaying and editing text, rather than displaying shapes on a visual design surface. 
Again the visual design surface as taught by Luo (as stated above) is simply the 
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background image portion of the character string that is extracted (each character 
represents an element with a shape). The Bates reference is simply utilized to show 
that it would be beneficial to visualize/display the extracted data of Luo to a user via 
bolding/highlighting the data of interest. 

Furthermore applicant has argued that since Luo is classified in class 382 (image 
analysis) and Bates is classified in class 715 (data processing) Luo and Bates are non- 
analogous art. Examiner respectfully disagrees. It should be noted that Luo teaches a 
method and system for detecting and extracting character data (via edge detection) and 
Bates teaches a method and system for detecting and subsequently displaying 
character data and therefore both references must are concerned with character data 
and both references require detecting of character data thus, may be constituted as 
analogous art. Again it should be noted that while Luo is concerned with image 
processing applications while Bates is concerned with text editing applications, Luo 
never necessitates nor implies that it is impossible to display the extracted data nor 
does Luo explicitly exclude the possibility of doing so in his reference. 

Lastly applicant has amended claims 1 and 12 specifying "associating the 
characteristic with an aura indicative of the determined value " and has argued that Luo 
and bates fail to teach or suggest these features. Examiner respectfully disagrees. It 
should be noted that two understandings of an aura "indicative of a determined value" 
can be interpreted. Firstly, an aura as taught by Luo is simply a bounding box bounding 
the area of the character and wherein each bounding box is associated with edge 
detection values as well as foreground/background contrast values. Therefore the aura 
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(bounding area) that is formed is indicative of the determined value (edge detection 
direction) of each candidate box (each character extracted) being a positive candidate 
(image data corresponding to character strings). (See p. 4 paragraphs 37-43). 
Therefore Luo teaches the aura of interest is indicative that the character string 
extracted is actually character data. Another interpretation of the said limitation is that 
Bates teaches displaying whether or not a particular word is repeated by detecting and 
subsequently highlighting/bolding such a word. Thus, the determined value as taught 
by Bates is whether or not the particular word is repeated or not and the aura indicates 
said value by displaying whether or not the word is highlighted or not. The amendment 
of claim 19 is similar in scope to claims 1 and 12 and thus the same rationale as noted 
above will be used in subsequent rejection. For rationale of rejection of new claim 21 as 
well as providing a reference with respect to applicant's challenge of official notice of 
claim 7, see below. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which fomis the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 1-4. 8-13, 19-21 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Luo (20030198386) in view of Bates (6976214) 

Regarding claim 1 , Luo teaches representing an element with a shape on a 
visual design surface. (Fig. 2, Figs. 8A-8B, p. 3 paragraph 35, p.4 paragraph 37, p. 4 
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paragraphs 37-38, paragraph 43, p.5 paragraphs 45-48, p. 6 paragraphs 49-50) It 
should be noted that the Luo teaches a character, which represents an element with a 
shape. (See. Fig. 5, Fig. 8B for example) Furthermore Luo teaches determining a value 
of a characteristic of the element. (Fig. 2, Figs. 8A-8B, p. 3 paragraph 35, p.4 
paragraph 37, p. 4 paragraphs 37-38, paragraph 43, p.5 paragraphs 45-48, p. 6 
paragraphs 49-50) It should be noted that Luo teaches for each character in the 
character string extracted, edge detection is utilized to determine whether a candidate 
box corresponds to a positive candidate (correspond to text regions) where edges are 
detected. Furthermore Luo teaches associating the characteristic with an aura 
indicative of the determined value. (Fig. 2, Figs. 8A-8B, p. 3 paragraph 35, p.4 
paragraph 37, p. 4 paragraphs 37-38, paragraph 43, p.5 paragraphs 45-48, p. 6 
paragraphs 49-50) It should be noted that the aura as taught by the Luo is bounding 
box bounding the area of the character and wherein each bounding box is associated 
with edge detection values as well as foreground/background contrast values, (p. 4 
paragraphs 37-38, paragraph 43, p.5 paragraphs 45-48, p. 6 paragraphs 49-50) 
Nonetheless, Luo does not explicitly teach displaying the aura in association with the 
shape on the design surface. This is what Bates teaches. (Figs. 4-6, Col 4 line 4-Col 5 
line 56) It should be noted that Bates teaches displaying text box in bold and 
highlighting the term in the text box. It would have been obvious to one of ordinary skill 
in the art at the time the invention was made to combine the teachings of displaying a 
term (word) inside a text box in bold and/or highlighting the term in the text box because 
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the user is provided with the additional functionality editing text and thus, improving 
composition and form of text. 

Regarding claim 19, Luo teaches a multiple design element shapes (character 
data with different shapes) and an aura in proximity to the design element shape to 
represent a value of a characteristic. (Fig. 2, Figs. 8A-8B, p. 3 paragraph 35. p.4 
paragraph 37, p. 4 paragraphs 37-38, paragraph 43, p.5 paragraphs 45-48, p. 6 
paragraphs 49-50) It should be noted that the aura as taught by the Luo is bounding 
box bounding the area of the character and wherein each bounding box is associated 
with edge detection values as well as foreground, background contrast values, (p. 4 
paragraphs 37-38, paragraph 43, p.5 paragraphs 45-48, p. 6 paragraphs 49-50) 
Nonetheless, Luo does not explicitly teach displaying the design shape element 
(character) and displaying the aura, the auras being displayed with different visual 
characteristics in order to indicate that the design element shapes have different values 
corresponding to particular characteristic. This is what Bates teaches. (Figs. 4-6, Col 4 
line 4-Col 5 line 56) It should be noted that Bates teaches displaying text box in bold 
and highlighting the term in the text box. It should be noted that Bates teaches different 
visual characteristics (indicating a particular word is highlighted or not) in order to 
indicate that the diesng element shapes (characters) have different values 
(repeated/non-repeated) corresponding to that characterisitic (highlighted/non- 
highlighted). It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of displaying a term (word) inside a text 
box in bold and/or highlighting the term in the text box because the user is provided with 
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the additional functionality editing text (such as that extracted in Luo) and thus, 
improving composition and form of text. 

Regarding claim 20, Luo teaches the aura is around the design element shape. 
(Fig. 2, Figs. 8A-8B, p. 3 paragraph 35, p.4 paragraph 37, p. 4 paragraphs 37-38, 
paragraph 43, p.5 paragraphs 45-48, p. 6 paragraphs 49-50) It should be noted that the 
aura as taught by the Luo is bounding box bounding the area of the character and 
wherein each bounding box is associated with edge detection values as well as 
foreground, background contrast values, (p. 4 paragraphs 37-38, paragraph 43, p.5 
paragraphs 45^8, p. 6 paragraphs 49-50) Nonetheless, Luo does not explicitly teach 
displaying the aura. This is what Bates teaches. (Figs. 4-6, Col 4 line 4-Col 5 line 56) It 
should be noted that Bates teaches displaying text box in bold and highlighting the term 
in the text box. It would have been obvious to one of ordinary skill in the art at the time 
the invention was made to combine the teachings of displaying a term (word) inside a 
text box in bold and/or highlighting the term in the text box because the user is provided 
with the additional functionality editing text and thus, improving composition and forni of 
text. 

Regarding claim 21 Bates teaches different visual characteristic associated with 
different potential values of the characteristic and displays the aura with visual 
characteristic associated with determined value of the characteristic. (Figs. 4-6, Col 4 
line 4-Col 5 line 56) The different visual characteristic (highlighted/non-highlighted) is 
associated with a different potential value (word being repeated or not, word needed to 
be replaced or not) as taught by Bates. It would have been obvious to one of ordinary 



Application/Control Number: 10/798,206 Page 8 

Art Unit: 2628 

skill in the art at the time the invention was made to combine the teachings of displaying 
a term (word) inside a text box in bold and/or highlighting the term in the text box 
because the user is provided with the additional functionality editing text and thus, 
improving composition and form of text. 

Regarding claim 2, Luo teaches the aura comprises a color coded area 
surrounding the shape. It should be noted that the image extracted is a gray scale 
image and the grayscale image is used to obtain edge representation of the image, (p. 2 
paragraph 27, p. 4 paragraph 36) 

Regarding claim 3, Luo teaches the aura comprises a color-coded area adjacent 
to at least a portion of the shape, (p. 2 paragraph 27, p. 4 paragraph 36, Fig. 2, Figs. 
8A-8B) 

Consider claim 4, Luo teaches repeating steps in claim 1 for a plurality of 
elements (i.e. additional characters of the extracted word, each character contained in a 
bounding box) (Fig. 2, Figs. 8A-8B, p. 3 paragraph 35, p.4 paragraph 37, p. 4 
paragraphs 37-38, paragraph 43, p.5 paragraphs 45-48, p. 6 paragraphs 49-50) 
Furthermore, Luo teaches merging aura associated with at least some elements in 
close proximity and having the same value characteristic. (Figs. 8A-86, p.5 paragraphs 
45-48, p. 6 paragraphs 49-50) It should be noted that the common value characteristic 
as taught by Luo for each element (character) is acceptable value of contrast between 
foreground and background, (p. 5 paragraph 46) Nonetheless, Luo does not explicitly 
teach displaying the aura in association with the shape on the design surface. This is 
what Bates teaches. (Figs. 4-6, Col 4 line 4-Col 5 line 56) It should be noted that Bates 
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teaches displaying text box in bold and highlighting the terni in the text box. It would 
have been obvious to one of ordinary sicill in the art at the time the invention was made 
to combine the teachings of displaying a term (word) inside a text box in bold and/or 
highlighting the term in the text box because the user is provided with the additional 
functionality editing text and thus, improving composition and fonn of text. 

Regarding claim 8, Luo teaches the at least one characteristic comprises a use 
for the element, (p.4 paragraph 37, p. 4 paragraphs 37-38, paragraph 43, p.5 
paragraphs 45-48, p. 6 paragraphs 49-50) It should be noted that the edge 
(characteristic) allows the character to be used (detected) within a word. 

Regarding claim 11, Luo teaches wherein at least one characteristic comprises 
an identification of an importance level, (p.4 paragraph 37, p. 4 paragraphs 37-38, 
paragraph 43, p.5 paragraphs 45-48, p. 6 paragraphs 49-50) It should be noted that 
text may be recognized as of higher importance level than non-text. 

Consider claim 12, Luo teaches determination a value of at least as a second 
characteristic of the element. (Fig. 2, Figs. 8A-8B, p. 3 paragraph 35, p.4 paragraph 37, 
p. 4 paragraphs 37-38, paragraph 43, p.5 paragraphs 45-48, p. 6 paragraphs 49-50) It 
should be noted that a second characteristic as taught by Luo may be contrast between 
foreground and background, (p.5 paragraphs 45-46) Associating the second 
characteristic with a second aura indicative of the value of the second characteristic is 
taught in Fig. 8B (Merging auras to create a word). Nonetheless, Luo does not explicitly 
teach displaying the second aura in association with the shape on the design surface. 
This is what Bates teaches. (Figs. 4-6, Col 4 line 4-Col 5 line 56) It should be noted that 
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Bates teaches displaying text box in bold and highlighting the temn in the text box. It 
would have been obvious to one of ordinary skill in the art at the time the invention was 
made to combine the teachings of displaying a term (word) inside a text box in bold 
and/or highlighting the temn in the text box because the user is provided with the 
additional functionality editing text and thus, improving composition and form of text. 

Claim 13 is similar in scope to claim 12 except for the recitation of determining a 
value of at least a third characteristic of the element. This is taught by Luo. (p.4 
paragraphs 39-42) It should be noted that the third characteristic as taught by Luo is 
direction information. Thus, claim 13 is rejected under similar rationale as claim 12. 

Regarding claim 9, Luo does not explicitly teach one characteristic comprises 
identification of namespace. Examiner takes official notice that extracted words may be 
names. It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to extract names into the combination of Luo and Bates because 
the user is provided with the additional functionality of editing names. 

Regarding claim 10, Luo does not explicitly teach one characteristic comprises 
identification of an application layer. Examiner takes official notice that images with 
some text may identify an application layer. It would have been obvious to one of 
ordinary skill in the art at the time the invention was made to provide highlights to those 
type of images to better visually identify and label said images. 

Claims 5-6 are rejected under 35 U.S.C. 103(a) as being unpatentable over Luo 
(20030198386) in view of Bates (6976214) in further view of Tappert. (4731857) 
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Regarding claim 5, Luo teaches repeating steps in claim 1 for a plurality of 
elements (i.e. additional characters of the extracted word, each character contained in a 
bounding box) (Fig. 2, Figs. 8A-8B, p. 3 paragraph 35, p.4 paragraph 37, p. 4 
paragraphs 37-38, paragraph 43, p.5 paragraphs 45-48, p. 6 paragraphs 49-50) For 
motivation of displaying the aura in association with shape, see claim 1. Nonetheless 
neither Luo nor Bates explicitly teaches sorting the elements (characters) so that 
elements having the same characteristics are generally in the same location. However, 
it should be noted that Luo does teach elements having the same characteristic 
(positive candidate bounding boxes) are generally located in the same region (see Fig. 
88, merging of bounding boxes [auras]). However Luo fails to teach sorting of 
characters. This is what Bates teaches. (Col 3 lines 7-19, Col 3 lines 48-68, Col 5 lines 
1-18, Col 6line 51-Col 8 line 29) It would have been obvious to one of ordinary skill in 
the art at the time the invention was made to sort characters for word recognition 
purposes into the system of Luo because initial character segmentation is not final and 
need not be highly accurate but is subject to lesser constraint of containing all the true 
segmentation points (Col 3 lines 64-68) and thus, a more efficient word recognition 
system can be achieved. 

Regarding claim 6, Luo teaches grouping elements in affinity regions, wherein 
each affinity region represents elements having the same value of the characteristic. 
(Fig. 2, Figs. 8A-8B, p. 3 paragraph 35, p.4 paragraph 37, p. 4 paragraphs 37-38, 
paragraph 43, p.5 paragraphs 45-48, p. 6 paragraphs 49-50) It should be noted that 
Luo teaches mergining two auras both having positive candidates (edge detected 
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and/or proper allowed constrast levels). For motivation of displaying elements in 
merged auras (affinity regions), see claim 1. 

Claim 7 is rejected under 35 U.S.C. 103(a) as being unpatentable over Luo 
(20030198386) in view of Bates (6976214) in further view of Tappert. (4731857) and 
Shostak (5893125) 

Regarding claim 7, it should noted that Examiner is providing Shostak as a 
reference due to applicant's challenge of examiner's official notice in the previous 
action. Shostak teaches utilizing labels such an image icon to indicate the status of an 
object or its contents. (Col 8 lines 2-33, Fig 3, Col 15 lines 6-19, Fig. 1 1 B) It should be 
noted that Shostak teaches that when the toolbar (label/icon) corresponding to selected 
text object is invoked (changes in status), having for changing the font of the selected 
text may be invoked as well. It would have been obvious to one of ordinary skill in the 
art at the time the invention was made to utilize a label to allow user to identify and 
change the status of a highlighted text box as taught by Shostak into the system of Luo 
because labels provide the functionality of giving users a convenient way of 
identification as well as describe characteristics of the object that would not be obvious 
from simple examination. 

Conclusion 

THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
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TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Kevin K. Xu whose telephone number is 571-272-7747. 
The examiner can normally be reached on 8:30AM - 5:00 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mark Zimmerman can be reached on 571-272-7653. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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information regarding the status of an application may be obtained from ti^e 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 




Kevin Xu 
7/31/07 
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SUPERVISORY PATENT EXAMINER 
TECHNOLOGY CENTER 2600 



